Zoom detection and fluoroscope movement detection for target overlay

ABSTRACT

Systems and methods for visualizing navigation of a medical device with respect to a target using a live fluoroscopic view. The methods include determining a level of zoom of a fluoroscope when acquiring a reference frame. The methods further include determining whether the live fluoroscopic images evidence movement of the fluoroscope relative to its position when capturing a reference frame.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims the benefit of and priority to U.S.Provisional Application Ser. No. 63/056,032, filed on Jul. 24, 2020, theentire contents of which are hereby incorporated by reference herein.

BACKGROUND Technical Field

This disclosure relates to the field of visualizing the navigation ofmedical devices, such as biopsy or ablation tools, relative to targets,confirming the relative positions, and monitoring for movement of thefluoroscope.

Description of Related Art

There are several commonly applied medical methods, such as endoscopicprocedures or minimally invasive procedures, for treating variousmaladies affecting organs including the liver, brain, heart, lungs, gallbladder, kidneys, and bones. Often, one or more imaging modalities, suchas magnetic resonance imaging (Mill), ultrasound imaging, computedtomography (CT), or fluoroscopy are employed by clinicians to identifyand navigate to areas of interest within a patient and ultimately atarget for biopsy or treatment. In some procedures, pre-operative scansmay be utilized for target identification and intraoperative guidance.However, real-time imaging may be required to obtain a more accurate andcurrent image of the target area. Furthermore, real-time image datadisplaying the current location of a medical device with respect to thetarget and its surroundings may be needed to navigate the medical deviceto the target in a safe and accurate manner (e.g., without causingdamage to other organs or tissue).

For example, an endoscopic approach has proven useful in navigating toareas of interest within a patient, and particularly so for areas withinluminal networks of the body such as the lungs. To enable the endoscopicapproach, and more particularly the bronchoscopic approach in the lungs,endobronchial navigation systems have been developed that use previouslyacquired Mill data or CT image data to generate a three-dimensional (3D)rendering, model, or volume of the particular body part such as thelungs.

The resulting volume generated from the MM scan or CT scan is thenutilized to create a navigation plan to facilitate the advancement of anavigation catheter (or other suitable medical device) through abronchoscope and a branch of the bronchus of a patient to an area ofinterest. A locating or tracking system, such as an electromagnetic (EM)tracking system, may be utilized in conjunction with, for example, CTdata, to facilitate guidance of the navigation catheter through thebranch of the bronchus to the area of interest. In certain instances,the navigation catheter may be positioned within one of the airways ofthe branched luminal networks adjacent to, or within, the area ofinterest to provide access for one or more medical instruments.

However, a 3D volume of a patient's lungs, generated from previouslyacquired scans, such as CT scans, may not provide a basis sufficient foraccurate guiding of medical devices or instruments to a target during anavigation procedure. In some cases, the inaccuracy is caused bydeformation of the patient's lungs during the procedure relative to thelungs at the time of the acquisition of the previously acquired CT data.This deformation (CT-to-Body divergence) may be caused by many differentfactors including, for example, changes in the body when transitioningfrom between a sedated state and a non-sedated state, the bronchoscopechanging the patient's pose, the bronchoscope pushing the tissue,different lung volumes (e.g., the CT scans are acquired during inhalewhile navigation is performed during breathing), different beds,different days, etc.

Thus, another imaging modality is needed to visualize medical devicesand targets in real-time and enhance the in-vivo navigation procedure.Furthermore, to accurately and safely navigate medical devices to aremote target, for example, for biopsy or treatment, both the medicaldevice and the target should be visible in a guidance system.

SUMMARY

This disclosure is directed to systems and methods for visualizingintra-body navigation of a medical device relative to a target using alive 2D fluoroscopic view. In accordance with the disclosure, one aspectis directed to method of determining a level of zoom in a fluoroscopicimage including capturing a fluoroscopic reference frame, calculating aninterval between radio opaque markers in the reference frame,calculating an expected interval between radio opaque markers for alllevels of zoom and identifying the expected interval that most closelymatches calculated interval in the reference frame.

In aspects, the method may further include storing a height of afluoroscope from which the fluoroscopic reference frame is capturedcompared to a location of the radio opaque markers.

In aspects, the method may further include storing an angle of thefluoroscope from which the fluoroscopic reference frame is capturedcompared to a location of the radio opaque markers.

In other aspects, the method may further include recalling the height ofthe fluoroscope.

In certain aspects, calculating the expected interval between radioopaque markers may include calculating the expected interval between theradio opaque markers at the recalled height of the fluoroscope at whichthe reference frame was captured.

In aspects, the method may further include applying a transform suchthat a location of a target can be accurately overlaid on a livefluoroscopic image.

In certain aspects, capturing the fluoroscopic reference frame mayinclude capturing the fluoroscopic reference frame when a fluoroscopefrom which the fluoroscopic reference frame is captured is in ananteroposterior position.

In other aspects, capturing the fluoroscopic reference frame may includecapturing the fluoroscopic reference frame through a sweep through about30-60 degrees relative to an anteroposterior position.

Another aspect of the disclosure is directed to a method of detectingmovement of a fluoroscope including capturing a reference frame,determining locations of radio opaque markers in the reference frame,creating a Gaussian mask, determining the locations of radio opaquemarkers in live fluoroscopic images, comparing the determined positionsof the radio opaque markers in the live fluoroscopic images to theGaussian mask, and determining whether the fluoroscope has movedrelative to its position when the reference frame was captured.

In aspects, determining the locations of the radio opaque markers mayinclude determining locations of the radio opaque markers in thereference frame using thresholding.

In other aspects, determining the locations of the radio opaque markersmay include determining a two-dimensional position of the radio opaquemarkers in a coordinate system of the fluoroscope.

In certain aspects, creating a gaussian mask may include creating aGaussian mask by defining an acceptable range of movement around thedetermined locations of the radio opaque markers.

In other aspects, the acceptable range of movement is an acceptablerange of movement in both an X and a Y direction in the reference frame.

In certain aspects, determining whether the fluoroscope has moved mayinclude determining whether the position of the radio opaque markers isoutside of the Gaussian mask.

Yet a further aspect of the disclosure is directed to a method includingacquiring a reference frame, determining a level of zoom in thereference frame, acquiring a position of a catheter, receiving a markedlocation of the catheter in the reference frame, generate a translationfrom the marked location and the detected position, acquire livefluoroscopic images; determining whether the fluoroscope has moved fromthe position at which the reference frame was acquired, and overlaying atarget identified in prior fluoroscopic images on the live fluoroscopicimages.

In aspects, acquiring the reference frame may include acquiring thereference frame through a sweep of the fluoroscope through about 30-60degrees relative to an anteroposterior position.

In other aspects, determining the level of zoom may include calculatingan interval between radio opaque markers in the reference frame,calculating an expected interval between the radio opaque markers forall levels of zoom, and identifying the expected interval that mostclosely matches the calculated interval in the reference frame.

In certain aspects, calculating the expected interval between radioopaque markers may include calculating the expected interval between theradio opaque markers at a recalled height and angle of the fluoroscoperelative to a location of the radio opaque markers.

In other aspects, determining whether the fluoroscope has moved mayinclude determining locations of radio opaque markers in the referenceframe, creating a Gaussian mask, determining the locations of the radioopaque markers in live fluoroscopic images, comparing the determinedpositions of the radio opaque markers in the live fluoroscopic images tothe Gaussian mask, and determining whether the fluoroscope has movedrelative to its position when the reference frame was acquired.

In certain aspects, determining whether the fluoroscope has movedincludes determining whether the position of the radio opaque markers isoutside of the Gaussian mask.

BRIEF DESCRIPTION OF THE DRAWINGS

Various aspects and embodiments of the disclosure are describedhereinbelow with references to the drawings, wherein:

FIG. 1 is a schematic diagram of a system for navigating to soft-tissuetargets via luminal networks in accordance with the disclosure;

FIG. 2 is a screen shot of an example user interface for confirminglocal registration in accordance with the disclosure;

FIG. 3 is a screen shot of an example navigation user interface with apop-up message inquiring about whether to continue with a target overlayfeature in accordance with the disclosure;

FIG. 4 is a screen shot of an example navigation user interface withouta target overlay feature in accordance with the disclosure;

FIG. 5 is a screen shot of an example navigation user interface with thetarget overlay feature in accordance with the disclosure;

FIG. 6 is a screen shot of an example navigation user interfaceillustrating a screen that appears when a “Target Overlay” tab isselected in accordance with the disclosure;

FIG. 7 is a flow diagram of an example method presenting a target on alive fluoroscopic image in accordance with the disclosure;

FIG. 8 is a screen shot of an example navigation user interface showinga target marker overlaid on a real-time two-dimensional (2D)fluoroscopic view in accordance with the disclosure;

FIG. 9 depicts three fluoroscopic images at different levels of zoom;

FIG. 10 is a flow diagram for determining the level of zoom as depictedin FIG. 9;

FIG. 11 is a flow diagram for determining whether a fluoroscope hasmoved as compared to a reference frame;

FIG. 12 is a graphical representation of the flow diagram of FIG. 11;

FIG. 13 is a flow diagram for a method of incorporating the methods ofFIGS. 10 and 11 to enable overlay of a previously identified target in alive fluoroscopic image;

FIG. 14 is a flow diagram of an example method of visualizing thenavigation of a medical device relative to a target; and

FIG. 15 is a schematic diagram of a system in accordance with thedisclosure for navigating to a target and displaying user interfaces inaccordance with the disclosure.

DETAILED DESCRIPTION

A fluoroscopic imaging device may be used by a clinician, for example,to visualize the navigation of a medical device and confirm theplacement of the medical device after it has been navigated to a desiredlocation. However, although fluoroscopic images show highly denseobjects, such as metal tools, bones, and large soft-tissue objects,e.g., the heart, the fluoroscopic images may not clearly show smallsoft-tissue objects of interest, such as lesions. Furthermore, thefluoroscopic images are two-dimensional projections. Therefore, an X-rayvolumetric reconstruction is needed to enable identification of softtissue objects and navigation of medical devices to those objects.

Several solutions exist that provide 3D volume reconstruction. Onesolution is a CT machine, which algorithmically combines multiple X-rayprojections from known, calibrated X-ray source positions into a 3Dvolume, in which soft tissues are more visible. For example, a CTmachine can be used with iterative scans during a procedure to provideguidance through the body until the tool or tools reach the target. Thisis a tedious procedure, as it requires several full CT scans, adedicated CT room, and blind navigation between scans. In addition, eachscan requires the staff to leave the room due to high levels of ionizingradiation and exposes the patient to the radiation. Another solution isa cone-beam CT machine. However, the cone-beam CT machine is expensiveand, like the CT machine, only provides blind navigation between scans,requires multiple iterations for navigation, and requires the staff toleave the room. In some example embodiments, the systems and methods ofthis disclosure combine the benefits of CT machines and fluoroscopicimaging devices to help clinicians navigate medical devices to targets,including small soft-tissue objects.

In an electromagnetic navigation procedure, planning, registration, andnavigation are performed to ensure that a medical device, e.g., a biopsytool, follows a planned path to reach a target, e.g., a lesion, so thata biopsy or treatment of the target can be completed. Following thenavigation phase, fluoroscopic images may be captured and utilized in alocal registration process to reduce CT-to-body divergence. After thelocal registration process, the locatable guide may be removed from theextended working channel and a medical device, e.g., a biopsy tool, isintroduced into the extended working channel and navigated to the targetto perform the biopsy or treatment of the target, e.g., the lesion.

In navigating the medical device to the target, clinicians may use alive 2D fluoroscopic view to visualize the position of the medicaldevice relative to the target. While the medical device may be clearlyvisible in the live fluoroscopic view, some targets, e.g., lesions, maynot be visible in the live fluoroscopic view. And electromagneticnavigation cannot be used because some medical devices do not includesensors. Moreover, the user interfaces that are used to advance ornavigate a medical device towards the target do not provide enoughinformation regarding the medical device relative to the target,including when the medical device is near the target.

This disclosure features a user interface which overlays a 2D targetmarker, which corresponds to a three-dimensional model of a targetidentified in a CT scan, on the live 2D fluoroscopic view so that aclinician can visualize the position of the medical device tip relativeto the target. Since the live fluoroscopic view with the target overlayis a two-dimensional view and does not necessarily show whether themedical device is above or below the target, the same user interfacealso includes a three-dimensional, medical device tip view of the 3Dmodel of the target, which enables a clinician to confirm that themedical device is not above or below the target.

The user interface also provides a graphical indication of whether themedical device is aligned with the target in three dimensions. Forexample, when the medical device is aligned in three-dimensions with thetarget, the user interface shows the target overlay in a first color,e.g., green. On the other hand, when the medical device is not alignedwith the target in three dimensions, the user interface shows the targetoverlay in a second color different from the first color, e.g., orangeor red.

In accordance with aspects of the disclosure, the visualization ofintra-body navigation of a medical device, e.g., a biopsy tool, towardsa target, e.g., a lesion, may be a portion of a larger workflow of anavigation system, such as an electromagnetic navigation system. FIG. 1is a perspective view of an exemplary system for facilitating navigationof a medical device, e.g., a biopsy tool, to a soft-tissue target viaairways of the lungs. System 100 may be further configured to constructfluoroscopic based three-dimensional volumetric data of the target areafrom 2D fluoroscopic images. System 100 may be further configured tofacilitate approach of a medical device to the target area by usingElectromagnetic Navigation Bronchoscopy (ENB) and for determining thelocation of a medical device with respect to the target.

One aspect of the system 100 is a software component for reviewing ofcomputed tomography (CT) image data that has been acquired separatelyfrom system 100. The review of the CT image data allows a user toidentify one or more targets, plan a pathway to an identified target(planning phase), navigate a catheter 102 to the target (navigationphase) using a user interface, and confirming placement of a sensor 104relative to the target. One such EMN system is the ELECTROMAGNETICNAVIGATION BRONCHOSCOPY® system currently sold by Medtronic PLC. Thetarget may be tissue of interest identified by review of the CT imagedata during the planning phase. Following navigation, a medical device,such as a biopsy tool or other tool, may be inserted into catheter 102to obtain a tissue sample from the tissue located at, or proximate to,the target.

As shown in FIG. 1, catheter 102 is part of a catheter guide assembly106. In practice, catheter 102 is inserted into a bronchoscope 108 foraccess to a luminal network of the patient P. Specifically, catheter 102of catheter guide assembly 106 may be inserted into a working channel ofbronchoscope 108 for navigation through a patient's luminal network. Alocatable guide (LG) 110, including a sensor 104 is inserted intocatheter 102 and locked into position such that sensor 104 extends adesired distance beyond the distal tip of catheter 102. The position andorientation of sensor 104 relative to the reference coordinate system,and thus the distal portion of catheter 102, within an electromagneticfield can be derived. Catheter guide assemblies 106 are currentlymarketed and sold by Medtronic PLC under the brand names SUPERDIMENSION®Procedure Kits, or EDGE™ Procedure Kits, and are contemplated as useablewith the disclosure.

System 100 generally includes an operating table 112 configured tosupport a patient P, a bronchoscope 108 configured for insertion throughpatient P's mouth into patient P's airways; monitoring equipment 114coupled to bronchoscope 108 (e.g., a video display, for displaying thevideo images received from the video imaging system of bronchoscope108); a locating or tracking system 114 including a locating module 116,a plurality of reference sensors 18 and a transmitter mat 120 includinga plurality of incorporated markers; and a computing device 122including software and/or hardware used to facilitate identification ofa target, pathway planning to the target, navigation of a medical deviceto the target, and/or confirmation and/or determination of placement ofcatheter 102, or a suitable device therethrough, relative to the target.Computing device 122 may be similar to workstation 1501 of FIG. 15 andmay be configured to execute the methods of the disclosure including themethod of FIGS. 10, 11, 13, and 14.

A fluoroscopic imaging device 124 capable of acquiring fluoroscopic orx-ray images or video of the patient P is also included in thisparticular aspect of system 100. The images, sequence of images, orvideo captured by fluoroscopic imaging device 124 may be stored withinfluoroscopic imaging device 124 or transmitted to computing device 122for storage, processing, and display. Additionally, fluoroscopic imagingdevice 124 may move relative to the patient P so that images may beacquired from different angles or perspectives relative to patient P tocreate a sequence of fluoroscopic images, such as a fluoroscopic video.The pose of fluoroscopic imaging device 124 relative to patient P andwhile capturing the images may be estimated via markers incorporatedwith the transmitter mat 120. The markers are positioned under patientP, between patient P and operating table 112 and between patient P and aradiation source or a sensing unit of fluoroscopic imaging device 124.The markers incorporated with the transmitter mat 120 may be twoseparate elements which may be coupled in a fixed manner oralternatively may be manufactured as a single unit. Fluoroscopic imagingdevice 124 may include a single imaging device or more than one imagingdevice.

Computing device 122 may be any suitable computing device including aprocessor and storage medium, wherein the processor is capable ofexecuting instructions stored on the storage medium. Computing device122 may further include a database configured to store patient data, CTdata sets including CT images, fluoroscopic data sets includingfluoroscopic images and video, fluoroscopic 3D reconstruction,navigation plans, and any other such data. Although not explicitlyillustrated, computing device 122 may include inputs, or may otherwisebe configured to receive, CT data sets, fluoroscopic images/video andother data described herein. Additionally, computing device 122 includesa display configured to display graphical user interfaces. Computingdevice 122 may be connected to one or more networks through which one ormore databases may be accessed.

With respect to the planning phase, computing device 122 utilizespreviously acquired CT image data for generating and viewing athree-dimensional model or rendering of patient P's airways, enables theidentification of a target on the three-dimensional model(automatically, semi-automatically, or manually), and allows fordetermining a pathway through patient P's airways to tissue located atand around the target. More specifically, CT images acquired fromprevious CT scans are processed and assembled into a three-dimensionalCT volume, which is then utilized to generate a three-dimensional modelof patient P's airways. The three-dimensional model may be displayed ona display associated with computing device 122, or in any other suitablefashion. Using computing device 122, various views of thethree-dimensional model or enhanced two-dimensional images generatedfrom the three-dimensional model are presented. The enhancedtwo-dimensional images may possess some three-dimensional capabilitiesbecause they are generated from three-dimensional data. Thethree-dimensional model may be manipulated to facilitate identificationof target on the three-dimensional model or two-dimensional images, andselection of a suitable pathway through patient P's airways to accesstissue located at the target can be made. Once selected, the pathwayplan, three-dimensional model, and images derived therefrom, can besaved and exported to a navigation system for use during the navigationphase(s). One such planning software is the ILLUMISITE® planning suitecurrently sold by Medtronic PLC.

With respect to the navigation phase, a six degrees-of-freedomelectromagnetic locating or tracking system 114, or other suitablesystem for determining location, is utilized for performing registrationof the images and the pathway for navigation, although otherconfigurations are also contemplated. Tracking system 114 includes thetracking module 116, a plurality of reference sensors 118, and thetransmitter mat 120 (including the markers). Tracking system 114 isconfigured for use with a locatable guide 110 and particularly sensor104. As described above, locatable guide 110 and sensor 104 areconfigured for insertion through catheter 102 into patient P's airways(either with or without bronchoscope 108) and are selectively lockablerelative to one another via a locking mechanism.

Transmitter mat 120 is positioned beneath patient P. Transmitter mat 120generates an electromagnetic field around at least a portion of thepatient P within which the position of a plurality of reference sensors118 and the sensor 104 can be determined with use of a tracking module116. A second electromagnetic sensor 126 may also be incorporated intothe end of the catheter 102. The second electromagnetic sensor 126 maybe a five degree-of-freedom sensor or a six degree-of-freedom sensor.One or more of reference sensors 118 are attached to the chest of thepatient P. The six degrees of freedom coordinates of reference sensors118 are sent to computing device 122 (which includes the appropriatesoftware) where they are used to calculate a patient coordinate frame ofreference. Registration is generally performed to coordinate locationsof the three-dimensional model and two-dimensional images from theplanning phase, with the patient P's airways as observed through thebronchoscope 108, and allow for the navigation phase to be undertakenwith precise knowledge of the location of the sensor 104, even inportions of the airway where the bronchoscope 108 cannot reach.

Registration of the patient P's location on the transmitter mat 120 maybe performed by moving sensor 104 through the airways of the patient P.More specifically, data pertaining to locations of sensor 104, whilelocatable guide 110 is moving through the airways, is recorded usingtransmitter mat 120, reference sensors 118, and tracking system 114. Ashape resulting from this location data is compared to an interiorgeometry of passages of the three-dimensional model generated in theplanning phase, and a location correlation between the shape and thethree-dimensional model based on the comparison is determined, e.g.,utilizing the software on computing device 122. In addition, thesoftware identifies non-tissue space (e.g., air filled cavities) in thethree-dimensional model. The software aligns, or registers, an imagerepresenting a location of sensor 104 with the three-dimensional modeland/or two-dimensional images generated from the three-dimension model,which are based on the recorded location data and an assumption thatlocatable guide 110 remains located in non-tissue space in patient P'sairways. Alternatively, a manual registration technique may be employedby navigating the bronchoscope 108 with the sensor 104 to pre-specifiedlocations in the lungs of the patient P, and manually correlating theimages from the bronchoscope to the model data of the three-dimensionalmodel.

Though described herein with respect to EMN systems using EM sensors,the instant disclosure is not so limited and may be used in conjunctionwith flexible sensor, ultrasonic sensors, or without sensors.Additionally, the methods described herein may be used in conjunctionwith robotic systems such that robotic actuators drive the catheter 102or bronchoscope 108 proximate the target.

Following registration of the patient P to the image data and pathwayplan, a user interface is displayed in the navigation software whichsets for the pathway that the clinician is to follow to reach thetarget. Once catheter 102 has been successfully navigated proximate thetarget as depicted on the user interface, the locatable guide 110 may beunlocked from catheter 102 and removed, leaving catheter 102 in place asa guide channel for guiding medical devices including withoutlimitation, optical systems, ultrasound probes, marker placement tools,biopsy tools, ablation tools (i.e., microwave ablation devices), laserprobes, cryogenic probes, sensor probes, and aspirating needles to thetarget. A medical device may be then inserted through catheter 102 andnavigated to the target or to a specific area adjacent to the target.

Prior to inserting the medical device through the catheter 102, a localregistration process may be performed for each target to reduce theCT-to-body divergence. In a capture phase of the local registrationprocess, a sequence of fluoroscopic images may be captured and acquiredvia fluoroscopic imaging device 124, optionally by a user and accordingto directions displayed via computing device 122. A fluoroscopic 3Dreconstruction may be then generated via computing device 122. Thegeneration of the fluoroscopic 3D reconstruction is based on thesequence of fluoroscopic images and the projections of structure ofmarkers incorporated with transmitter mat 120 on the sequence of images.One or more slices of the 3D reconstruction may be then generated basedon the pre-operative CT scan and via computing device 122. The one ormore slices of the 3D reconstruction and the fluoroscopic 3Dreconstruction may be then displayed to the user on a display viacomputing device 122, optionally simultaneously. The slices of 3Dreconstruction may be presented on the user interface in a scrollableformat where the user is able to scroll through the slices in series.

In a marking phase of the local registration process, the clinician maybe directed to identify and mark the target while using the slices ofthe 3D reconstruction as a reference. The user may also be directed toidentify and mark the navigation catheter tip in the sequence offluoroscopic 2D images. An offset between the location of the target andthe navigation catheter tip may be then determined or calculated viacomputer device 122. The offset may be then utilized, via computingdevice 122, to correct the location and/or orientation of the navigationcatheter on the display (e.g., in the peripheral navigation screen whichmay be viewed by selecting the “Peripheral Navigation” tab 401illustrated in FIG. 4) with respect to the target and/or correct theregistration between the three-dimensional model and tracking system 114in the area of the target and/or generate a local registration betweenthe three-dimensional model and the fluoroscopic 3D reconstruction inthe target area.

In a confirmation phase of the local registration process, afluoroscopic 3D reconstruction is displayed in a confirmation screen202, which is illustrated in FIG. 2. The confirmation screen 202includes a slider 208 that may be selected and moved by the user toreview a video loop of the fluoroscopic 3D reconstruction, which showsthe marked target and navigation catheter tip from differentperspectives. After confirming that there are marks on the target andnavigation catheter tip throughout the video, the clinician may selectthe “Accept” button 210, at which point the local registration processends and the position of the navigation catheter is updated. Theclinician may then use the navigation views in, for example, theperipheral navigation screen illustrated in FIG. 4 to fine tune thealignment of the navigation catheter to the target before beginning anendoscopic procedure.

After the local registration process, the clinician or robot may inserta medical device in the catheter 102 and advance the medical devicetowards the target. While advancing the medical device towards thetarget, the clinician may view a user interface screen which includes:(a) a 3D medical device tip view of a 3D model of a target based onpre-operative CT scans, and (b) a live 2D fluoroscopic view on which atarget marker corresponding to the 3D model of the target is overlaid.This user interface screen allows the clinician to not only see themedical device in real-time, but also allows the clinician to seewhether the medical device is aligned with the target. The userinterface screen may also provide a graphical indication of whether themedical device is aligned in three-dimensions with the target. Forexample, when the medical device is aligned in three-dimensions with thetarget, the user interface shows the target overlay in a first color,e.g., green. On the other hand, when the medical device is not alignedwith the target in three dimensions, the user interface shows the targetoverlay in a second color different from the first color, e.g., orangeor red.

FIG. 2 is a screen shot of a confirmation screen 202 of an example localregistration user interface that appears during the confirmation phaseof the local registration process. The confirmation screen 202 displaysthe navigation catheter tip mark 204 and the target mark 206, which werepreviously marked by the clinician during the marking phase of the localregistration process. After the clinician selects the “Accept” button,the navigation user interface of FIG. 3 is displayed with a pop-upmessage 302. The pop-up message 302 may include buttons 304, 306, whichenable the clinician to select whether to use the target overlay featureto guide navigation of a medical device, e.g., a biopsy tool, to thetarget. Specifically, the clinician may select button 304 to continuewith the target overlay feature or the clinician may select button 306to continue without using the target overlay feature.

When the clinician selects button 306, the peripheral navigation screenassociated with the “Peripheral Navigation” tab 401 of the userinterface 400 of FIG. 4, which was previously displayed prior toperforming the local registration process, is redisplayed showingadjustments, if any, to the position and/or orientation of thenavigation catheter tip 405 as a result of the location registrationprocess. The peripheral navigation screen 401 includes a local CT view402, a 3D navigation catheter tip view 404, a 3D map view 406, and abronchoscope view 408. The peripheral navigation screen 401 alsoincludes a local registration user controls 403 enabling the user toapply local registration and/or relaunch local registration. The userinterface 400 also includes a “Central Navigation” tab 411 and a “TargetAlignment” tab 412, which may be individually selected to performcentral navigation or target alignment, respectively.

When the clinician selects button 403, the user interface 400 displaysthe peripheral navigation screen and a “Target Overlay” tab 502illustrated in FIG. 5. When the target overlay tab 502 is selected, atarget overly screen is displayed as seen in FIG. 6 which includes alive fluoroscopic view 602 of the catheter 102 in the patient “P”.

As illustrated in FIG. 7, following selection of the target alignmenttab 502 at step 702, a live fluoroscopic image 602 (FIG. 6) is displayedat step 704. Once the live fluoroscopic image 602 is displayed, thecomputing device 122 may request marking of the position of the tip 604of catheter 102 in the fluoroscopic image at step 706. As analternative, the marking of the tip 602 may be automatically performedby an image analysis application and may optionally be presented to auser to simply confirm. For example, this may be based on identifyingall the pixels in the image that have above a certain thresholdHounsfield unit value, which will be the radio opaque catheter. The lastconnected pixel of the pixels making up the catheter 102 will be the tip602 of the catheter 102. Of course, other processes may also be used forcatheter 102 tip 602 detection without departing from the scope of thedisclosure.

At step 708, following receipt of the position of the tip 604 of thecatheter in the fluoroscopic image 602, the computing device performs animage analysis of the live fluoroscopic image 602, and the fluoroscopic3D reconstruction acquired during the local registration process. Theimage processing at step 708 compares the live 2D fluoroscopic image 602to slices of the fluoroscopic 3D reconstruction to determine a bestmatch. This best match may be based on the shape of the catheter 102 ineach of the slices of the fluoroscopic 3D reconstruction as compared tothe shape of the catheter 102 in the live fluoroscopic image.Additionally or alternatively, the best match may be based on the markedposition of the tip 604 of the catheter 102 in the live fluoroscopicimage 602, and that marked during the local registration process. Stillfurther, the computing device 122 may compare the detected position ofthe catheter 102 by the tracking system 114 immediately following thelocal registration to the current detected position of the catheter 102.Any of these techniques may also be used in combination to assist in thematching. Although generally described as comparing the live 2Dfluoroscopic image 602 to slices of the fluoroscopic 3D reconstructionto determine a best match, it is contemplated that any suitable means ofidentifying a best match of the live fluoroscopic image 602 to imagescaptured during local registration may be utilized. In embodiments, theposition of the fluoroscopic imaging device 124 relative to the grid ofradio opaque markers 612 adjacent the transmitter mat 120 may beidentified and the location of the tip 604 of the catheter 102 can beutilized to identify the location of the fluoroscopic imagine device 124in the same dimensional space as the tip 604 of the catheter 102 (e.g.,the same coordinate system).

Following the image analysis, the computing device displays a locationof the target, whose position relative to the catheter 102 wasdetermined during the local registration and displays a target marker606 on the live fluoroscopic image 602 at step 710 as depicted in FIG.8. This position of the target marker 606 is based on the location ofthe target that was marked during the local registration processemploying the fluoroscopic 3D reconstruction.

In the example of FIG. 8, the tip 604 of the catheter 102 is shown asaligned with the target marker 606 which is displayed in the livefluoroscopic view 602. In addition, a medical device tip view 608depicts the view as if a camera were located at the tip 604 of thecatheter 102. The medical device tip view 608 presents athree-dimensional representation 610 of the target 606. If the tip 604of the catheter 102 is nearly aligned with the target, the target marker606 may be displayed in a first color (e.g., green) and overlaid on thelive fluoroscopic view 602. If, on the other hand, the tip 604 of thecatheter 102 were not aligned with the target (for example, as shown inthe 3D navigation catheter tip view 404 of FIG. 4 in which the sphericaltarget is disposed to the right of the center of the 3D navigationcatheter tip view 404), the target marker 606 may be displayed in adifferent color, e.g., orange or red. Similarly, in the medical devicetip view 608 where the tip 604 of the catheter 102 is not aligned withthe target, the 3D representation 610 of the target 606 will appearoffset in the image, and only a portion or none of it may be visible inthat view. In addition, the color may also change as described abovedepending on the severity of the misalignment of the tip 604 of thecatheter 102 and the target

The medical device tip view 608 may also include a text box 612, whichdisplays text indicating a distance between the tip 604 of the catheter102 and the center of target. In embodiments, the computing device 122may calculate the distance by aligning or finding the correspondencebetween the 3D model of the luminal network, which may be based on a CTscan and which includes the target, and the live fluoroscopic view, andmeasuring the distance between the tip 604 and the center of the 3Dmodel of the target using, for example, image processing. In finding thecorrespondence between the 3D model and the live fluoroscopic view, thefluoroscopic 3D reconstruction generated and marked in the localregistration process may be used. In embodiments, the distance ismeasured from a center of or an outside edge of the target. The targetoverlay screen further includes a target overlay toggle button 614,which, when selected, toggles between displaying the target marker 606as shown in FIG. 8 and not displaying the target marker 606.

While FIG. 7 depicts the general overview of a target overlay method andthe result of the target overlay. While effective, the capabilities ofthe fluoroscope and actions which occur during a procedure may requirefurther refinements of the process. One of the challenges that can occurwhen using a fluoroscope and comparing or importing aspects from one setof images to another is determining the level of zoom of a given image.Fluoroscopes generally have three levels of zoom. As depicted in FIG. 9,one level is zoom out (e.g., no zoom), a second is normal, and the thirdis zoom in. During a procedure, it is not uncommon for a clinician toperform the local registration, described above, at one level of zoom,for example zoom out, and then when seeking to perform the biopsy movethe fluoroscope to a normal or zoom in level to provide greater clarityof the procedure in the live images. In FIG. 9 the difference in zoomlevel can be observed by comparing the interval between the radio opaquemarkers 612 at each level of zoom. The catheter 102 is easily observablein each image of FIG. 9, however, as will be appreciated, by changingthe level of zoom, an overlaid position of a target captured at a normallevel of zoom might be overlaid in an entirely improper position if thelevel of zoom for the live fluoroscopic images is changed to zoom out orzoom in levels.

FIG. 10 is a flow chart that the computing device 122 may undertake todetermine the level of zoom of the fluoroscope. At step 1002 a referenceframe is captured by the fluoroscope 124. The reference frame is a livefluoroscopic image. At step 1004 the distance or interval between tworadio opaque markers 612 is calculated. In one example, the interval iscalculated between two for a center column of radio opaque markers 612.At step 1006 the, the height of the fluoroscope 124 from the transmittermat 120 on which the radio opaque markers 612 are incorporated, whichwas determined as part of the local registration is recalled from memorywithin the computing device 122. The height of the fluoroscope 124 isdetermined while at a zoom-out configuration. As can be appreciated, theheight of the fluoroscope 124 relative to the transmitter mat 120 mayhave a similar effect as increasing or decreasing the level of zoom.Therefore, it contemplated that the height of the fluoroscope 124 ismaintained during the procedure to ensure that the level of zoom is notaltered by a fluctuation of the height of the fluoroscope 124. At step1008 a simulation is performed to calculate an expected interval betweenthe radio opaque markers 612 at the height and angle of the fluoroscope124 at which the reference frame was captured for all possible zoomoptions. At step 1010 the zoom level in which the calculated intervalbetween the radio opaque markers 612 most closely matches the determinedinterval of the radio opaque markers 612 in the reference frame isidentified as the zoom level of the reference frame. With the zoom levelof the reference frame determined the computing device can apply atransform at step 1012 such that the location of the target, identifiedduring the local registration can be accurately overlaid on the livefluoroscopic image. In one non-limiting embodiment, the zoom level maybe calculated using the known height of the fluoroscope 124 and thecalculated interval between radio opaque markers 612. In this manner,rather than selecting the zoom level that most closely matches thecalculated interval between the radio opaque markers 612, the zoom levelmay be estimated or a zoom level that falls between the zoom out, normalzoom, and zoom in positions may be utilized. As can be appreciated,estimating or calculating the zoom interval reduces computational time,reduces the energy required to make the computation, and may support anoption where a continuous zoom level may be utilized (e.g., no discretezoom levels but rather a fluidly changing zoom level).

It has been determined that this process can be employed when thefluoroscope 124 is in the AP position as well as at angles to the APposition without impeding its utility. The fluoroscopic sweep for localregistration may be taken through about 30-60 degrees to generate the 3Dvolumetric reconstruction. In embodiments, the fluoroscopic sweep forlocal registration may be taken through a range of about 200 degrees.Zoom detection from at least about +45 to −45 degrees has been found tobe equally effective regardless of the angle of the fluoroscope 124 whendetermining the level of zoom.

A second aspect of use of a fluoroscope 124 for acquisition of biopsiesthat needs to be accounted for is the issue of movement of thefluoroscope 124. In general, there are two types of movement intentionalmovements and unintentional movements. Most unintentional movements maybe caused by a doctor or nurse inadvertently bumping the fluoroscope124, whereas intentional movements are those that are intended by thedoctor. It is these intentional movements that this instant aspect ofthe disclosure is intended to detect.

As FIG. 11 details a process 1100 for determining whether the abovedescribed intentional movements of the fluoroscope 124 have occurred. Aninitial step of movement detection, a reference frame is captured atstep 1102. This may in fact be the same reference frame that wascaptured at step 1002 of the zoom detection. At step 1104 the locationsof the radio opaque markers 612 in the reference frame are determined.This may be performed for example via thresholding or other imageprocessing techniques to identify the locations of the radio opaquemarkers 612 in the reference frame. These determined positions of theradio opaque markers 612 are the two-dimensional positions of the radioopaque markers 612 in the fluoroscope coordinate system.

Once the positions of the radio-opaque markers 612 are determined, aGaussian mask is created at step 1106. The Gaussian mask defines anacceptable range around the detected location of the radio opaquemarkers 612. As depicted in FIG. 12, the Gaussian mask defines anacceptable rage of movement in both the X and the Y directions. FIG. 12graphically depicts the steps outlined in FIG. 11 thus the samereference numerals are used to depict the steps described with respectto FIG. 11 as they are shown in FIG. 12.

Once the Gaussian mask is defined at step 1106, for each livefluoroscopic image (e.g., each frame of a fluoroscopic video) acquiredby the fluoroscope 124 while the clinician is seeking to perform abiopsy or other procedure, at step 1108 again the position of the radioopaque markers 612 is determined in the same manner as in the referenceframe at step 1104. The position of the radio opaque markers is thencompared at step 1110 to the Gaussian mask. If the position of the radioopaque markers 612 is outside of the Gaussian mask, the determination ismade at step 1112 it is an indicator that for that frame or single imagethe fluoroscope 124 has moved. If position of the radio opaque markers612 is inside of the Gaussian mask, the determination is made at step1114 that for that frame or single image the fluoroscope 124 has notmoved. Although generally referred to as being a Gaussian mask, it iscontemplated that any suitable method of analyzing location of the radioopaque markers 612 within a fluoroscopic image may be utilized.

FIG. 13 depicts a work-flow 1300 incorporating the zoom determinationand the movement detection principals described herein. In accordancewith the work flow a reference frame is captured at step 1302. Followingcapture of the reference frame zoom detection is undertaken at step1304. This zoom detection is in accordance with the procedure outlinedwith respect to FIG. 10. Next, at step 1306, the position of the radioopaque markers 612 in the reference image and generation of the Gaussianmask is undertaken. At step 1308 the clinician is requested to mark thedistal end of the catheter 102. At the same time the computing device122 acquires the electromagnetic position of the distal end of thecatheter 102 from the sensor 104 or 126 and the computing device 122calculates a translation or position of the fluoroscopic imaging device124 in the navigation coordinate system (e.g., antenna coordinates). Ascan be appreciated, with the height of the fluoroscope 124 known, thetranslation may be calculated for the remaining two coordinates orpositional indicators of the catheter 102 using the position of thedistal end of the catheter 102. With the X, Y, and Z coordinates known,the position of the distal end of the catheter 102 can be determined. Ascan be appreciated, knowing the position of the fluoroscope 124 in thenavigation coordinate system enables the estimated target location to beaccurately projected onto the live fluoroscopic image.

Once complete, the live images from the fluoroscope 124 are analyzed bythe process described in connection with FIG. 11 regarding determinationof movement at step 1310. As a result, when it is determined that therehas been no movement or at least no movement over the thresholds definedby the Gaussian mask, the target location of the target identified inthe local registration can then be projected on the live fluoroscopicimages at step 1312. If the determination is made that the fluoroscope124 has moved an amount greater than the threshold amounts defined bythe Gaussian masks from when the reference image was acquired, theentire process can revert back to step 1302 and require the acquisitionof a new reference image. The determination of movement for each frameof the fluoroscopic images is stored in memory. In one embodiment, asdescribed in greater detail below, a determination to cease targetoverlay of step 1312 may be made when a threshold number of images(e.g., successive images) indicate that the fluoroscope 124 has moved.

Though in its simplest from a single image determined at step 1310 torepresent that the fluoroscope 124 has moved could result in cessationof the process and require acquisition of a new reference image theinstant disclosure is not so limited. As will be appreciated, there areinstances where the fluoroscope 124 may be bumped or disturbed by adoctor or nurse during the procedure. In such instances, because thefluoroscope may oscillate following being disturbed, each frame may notbe found to depict the radio opaque markers 612 outside of thethresholds set by the Gaussian mask. Further, the oscillations maybecome smaller and eventually cease after period of time. In suchsituations, where ultimately the fluoroscope 124 has not beenintentionally moved, there is no need to have the process revert to step1302.

Similarly, there are instances where a doctor or nurse may adjust thecontrast or brightness of the fluoroscope 124. The result is that for agiven number of frames the radio opaque markers may not be identifiable.These frames do not indicate that there was intentional movement of thefluoroscope 124, but rather some other basis for the inability todetermine that the radio opaque markers are within the Gaussian mask.Again, after a period of time, or a number of frames, the radio opaquemarkers 612 will again be viewable, and the position of the fluoroscoperelative to its position when the reference frame was acquired can beconfirmed as unchanged.

In contrast, when a clinician has intentionally moved the fluoroscope124 from the position at which the reference frame was acquired, eachframe captured by the fluoroscope will have the positions of radioopaque markers fall outside of the Gaussian mask. Once sufficient framesare determined to be outside of the mask target overlay of step 1312 canbe ceased and the process returns to step 1302.

FIG. 14 is a flow diagram of an example method 1400 of visualizing thenavigation of the medical device tip towards a target after the medicaldevice tip is brought into the vicinity of the target. At block 1401,following navigation proximate the target, a local registration processis performed to update and the target position in the navigation plan.Following local registration, at block 1402, the computing device 122determines whether the “Target Overlay” tab 502 has been selected. Whenthe “Target Overlay” tab 502 is selected, the computing device 122determines at block 1403 the relative position of a tip 602 of catheter102 in a live fluoroscopic image and a target which was identifiedduring the local registration process and displays the target at therelative position in the live fluoroscopic image. This process isperformed for example using the steps described above with respect toFIG. 7.

Next, the computing device determines whether the medical device tip isaligned with the target at block 1404. The computing device 122 maydetermine whether the medical device tip is aligned with the target byaligning or finding the correspondence between the 3D model of theluminal network, which may be based on a CT scan and which includes thetarget, and the live fluoroscopic view; and determining whether themedical device tip is aligned with the 3D model of the target based onthe determined alignment or correspondence and applying, for example,image processing. In aligning or finding the correspondence between the3D model and the live fluoroscopic view, the fluoroscopic 3Dreconstruction generated and marked in the local registration processmay be used.

When the computing device 122 determines that the medical device tip isaligned with the target, the computing device 122 sets the target markcolor to green at block 1406; otherwise, the computing device 122 setsthe target mark color to orange at block 1408. At block 1410, thecomputing device 122 displays, in a target overlay screen, a live 2Dfluoroscopic view, which at least shows the medical device. At block1412, the computing device 122 displays a target mark having the setcolor overlaid on the live 2D fluoroscopic view. At block 1414, thecomputing device 122 displays, in the same target overlay screen, a 3Dvirtual target, which corresponds to the target mark, from theperspective of the medical device tip. Blocks 1404-1414 may be repeateduntil the medical device tip is placed at the center of the target oruntil the biopsy or other treatment is completed. This final navigationallows the user to use fluoroscopic navigation techniques to obtain liveimages with the target marked on the live images, which enables the userto see how well the medical device tip is aligned with the target toensure that the medical device tip reaches the target to take a sampleof the target or perform treatment on the target.

In a further aspect of the disclosure, following the local registration(e.g., step 901) the computing device 122 may undertake an imageanalysis of the fluoroscopic 3D reconstruction to determine an angle forplacement of the fluoroscopic imaging device 124 to optimally engage inthe target overlay tab. In this aspect of the disclosure, following theidentification of the tip of the catheter 102 in two slices of thefluoroscopic 3D reconstruction, and identification of the target in thefluoroscopic 3D reconstruction and determining the relative position ofthe tip of the catheter 102 and the target in the fluoroscopic 3Dreconstruction, the computing device performs an image analysis of the3D reconstruction to determine a slice of the 3D reconstruction at whichthe catheter and the target visible. This may be the slice where boththe target and the catheter 102 are most visible or most visible beyondsome minimum threshold. Those of skill in the art will appreciate thatthere will be slices in which one or the other (or both) of the catheteror target are not visible and those images will likely be ignored by thecomputing device 122 when performing this analysis.

After analyzing the remaining slices of the fluoroscopic 3Dreconstruction, one of the slices is identified as most clearlydepicting both the catheter 102 and the target. Once the slice of the 3Dreconstruction is determined, the position (e.g., angle to the patient Por operating table 112) of the fluoroscopic imaging device 124 where acorresponding 2D fluoroscopic image, such as image 602, can be captured.This position of the fluoroscopic imaging device 124 can be presented tothe clinician on a user interface prior to engaging the “Target Overlay”tab 502, so that they can manually move the fluoroscopic imaging device124 to that position. Alternatively, the fluoroscopic imaging device 124may receive an indication of the position determined by the computingdevice 122 and automatically drive the fluoroscopic imaging device tothat position such that upon selecting the “Target Overlay” tab 502 thefluoroscopic image 602 is acquired at this pre-determined optimumposition for viewing the catheter 102 and target.

Another aspect of the disclosure is the enablement of the use of zoomfeatures which may be built into the fluoroscopic imaging device 124. Asdepicted in FIG. 6, the live 2D fluoroscopic image 602 includes aplurality of radio opaque markers 612. These radio opaque markers 612may be placed on or embedded in the transmitter mat 120. The distancesbetween the radio opaque markers is fixed and known by the computingdevice 122. Because the distances between the radio opaque markers 612is known, if the distance between any of the markers exceeds the knowndistances the computing device 122 can determine that the zoom featuresof the fluoroscopic imaging device 124 are engaged. The exact amount ofzoom that has been engaged can be determined by comparing the spacing ofradio opaque markers 612 in the 2D fluoroscopic image 602 to the knownspacing of the radio opaque markers 612 in the transmitter mat 120. Oncethe amount of zoom is determined, the computing device can calculate anoffset in the relative position of the tip 604 of the catheter 102 andthe target such that the target marker 606 can be accurately displayedin the fluoroscopic image 602 despite the change in zoom from when thelocal registration process was undertaken.

Reference is now made to FIG. 15, which is a schematic diagram of asystem 1000 configured for use with the methods of the disclosureincluding the method of FIG. 14. System 1500 may include a workstation1501, and optionally a fluoroscopic imaging device or fluoroscope 1515.In some embodiments, workstation 1501 may be coupled with fluoroscope1515, directly or indirectly, e.g., by wireless communication.Workstation 1501 may include a memory 1502, a processor 1504, a display1506 and an input device 1510. Processor or hardware processor 1504 mayinclude one or more hardware processors. Workstation 1501 may optionallyinclude an output module 1512 and a network interface 1008. Memory 1502may store an application 1518 and image data 1514. Application 1518 mayinclude instructions executable by processor 1504 for executing themethods of the disclosure including the method of FIGS. 10, 11, 13 and14.

Application 1518 may further include a user interface 1516. Image data1514 may include the CT scans, the generated fluoroscopic 3Dreconstructions of the target area and/or any other fluoroscopic imagedata and/or the generated one or more slices of the 3D reconstruction.Processor 1504 may be coupled with memory 1502, display 1506, inputdevice 1510, output module 1512, network interface 1508 and fluoroscope1515. Workstation 1501 may be a stationary computing device, such as apersonal computer, or a portable computing device such as a tabletcomputer. Workstation 1501 may embed a plurality of computer devices.

Memory 1502 may include any non-transitory computer-readable storagemedia for storing data and/or software including instructions that areexecutable by processor 1504 and which control the operation ofworkstation 1501 and, in some embodiments, may also control theoperation of fluoroscope 1515. Fluoroscope 1515 may be used to capture asequence of fluoroscopic images based on which the fluoroscopic 3Dreconstruction is generated and to capture a live 2D fluoroscopic viewaccording to this disclosure. In an embodiment, memory 1502 may includeone or more storage devices such as solid-state storage devices, e.g.,flash memory chips. Alternatively, or in addition to the one or moresolid-state storage devices, memory 1502 may include one or more massstorage devices connected to the processor 1504 through a mass storagecontroller (not shown) and a communications bus (not shown).

Although the description of computer-readable media contained hereinrefers to solid-state storage, it should be appreciated by those skilledin the art that computer-readable storage media can be any availablemedia that can be accessed by the processor 1504. That is, computerreadable storage media may include non-transitory, volatile andnon-volatile, removable and non-removable media implemented in anymethod or technology for storage of information such ascomputer-readable instructions, data structures, program modules orother data. For example, computer-readable storage media may includeRAM, ROM, EPROM, EEPROM, flash memory or other solid-state memorytechnology, CD-ROM, DVD, Blu-Ray or other optical storage, magneticcassettes, magnetic tape, magnetic disk storage or other magneticstorage devices, or any other medium which may be used to store thedesired information, and which may be accessed by workstation 1501.

Application 1518 may, when executed by processor 1504, cause display1506 to present user interface 1516. User interface 1516 may beconfigured to present to the user a single screen including athree-dimensional (3D) view of a 3D model of a target from theperspective of a tip of a medical device, a live two-dimensional (2D)fluoroscopic view showing the medical device, and a target mark, whichcorresponds to the 3D model of the target, overlaid on the live 2Dfluoroscopic view, as shown, for example, in FIG. 8. User interface 1516may be further configured to display the target mark in different colorsdepending on whether the medical device tip is aligned with the targetin three dimensions.

Network interface 1508 may be configured to connect to a network such asa local area network (LAN) consisting of a wired network and/or awireless network, a wide area network (WAN), a wireless mobile network,a Bluetooth network, and/or the Internet. Network interface 1508 may beused to connect between workstation 1501 and fluoroscope 1515. Networkinterface 1508 may be also used to receive image data 1514. Input device1510 may be any device by which a user may interact with workstation1501, such as, for example, a mouse, keyboard, foot pedal, touch screen,and/or voice interface. Output module 1512 may include any connectivityport or bus, such as, for example, parallel ports, serial ports,universal serial busses (USB), or any other similar connectivity portknown to those skilled in the art. From the foregoing and with referenceto the various figures, those skilled in the art will appreciate thatcertain modifications can be made to the disclosure without departingfrom the scope of the disclosure.

While detailed embodiments are disclosed herein, the disclosedembodiments are merely examples of the disclosure, which may be embodiedin various forms and aspects. For example, embodiments of anelectromagnetic navigation system, which incorporates the target overlaysystems and methods, are disclosed herein; however, the target overlaysystems and methods may be applied to other navigation or trackingsystems or methods known to those skilled in the art. Therefore,specific structural and functional details disclosed herein are not tobe interpreted as limiting, but merely as a basis for the claims and asa representative basis for teaching one skilled in the art to variouslyemploy the disclosure in virtually any appropriately detailed structure.

While several embodiments of the disclosure have been shown in thedrawings, it is not intended that the disclosure be limited thereto, asit is intended that the disclosure be as broad in scope as the art willallow and that the specification be read likewise. Therefore, the abovedescription should not be construed as limiting, but merely asexemplifications of embodiments. Those skilled in the art will envisionother modifications within the scope and spirit of the claims appendedhereto.

1. A method of determining a level of zoom in a fluoroscopic image,comprising: capturing a fluoroscopic reference frame; calculating aninterval between radio opaque markers in the reference frame;calculating an expected interval between radio opaque markers for aplurality of levels of zoom; and identifying the expected interval thatmost closely matches the calculated interval in the reference frame. 2.The method according to claim 1, further comprising storing a height ofa fluoroscope from which the fluoroscopic reference frame is capturedcompared to a location of the radio opaque markers.
 3. The methodaccording to claim 2, further comprising storing an angle of thefluoroscope from which the fluoroscopic reference frame is capturedcompared to a location of the radio opaque markers.
 4. (canceled)
 5. Themethod according to claim 3, wherein calculating the expected intervalbetween radio opaque markers includes calculating the expected intervalbetween the radio opaque markers at the stored height of the fluoroscopeat which the reference frame was captured.
 6. The method according toclaim 5, further comprising applying a transform such that a location ofa target can be accurately overlaid on a live fluoroscopic image.
 7. Themethod according to claim 1, wherein capturing the fluoroscopicreference frame includes capturing the fluoroscopic reference frame whena fluoroscope from which the fluoroscopic reference frame is captured isin an anteroposterior position.
 8. The method according to claim 1,wherein capturing the fluoroscopic reference frame includes capturingthe fluoroscopic reference frame through a sweep through about 30-60degrees relative to an anteroposterior position.
 9. A method ofdetecting movement of a fluoroscope, comprising: capturing a referenceframe; determining locations of radio opaque markers in the referenceframe; creating a Gaussian mask; determining the locations of radioopaque markers in live fluoroscopic images; comparing the determinedpositions of the radio opaque markers in the live fluoroscopic images tothe Gaussian mask; and determining whether the fluoroscope has movedrelative to its position when the reference frame was captured.
 10. Themethod according to claim 9, wherein determining the locations of theradio opaque markers includes determining locations of the radio opaquemarkers in the reference frame using thresholding.
 11. The methodaccording to claim 9, wherein determining the locations of the radioopaque markers includes determining a two-dimensional position of theradio opaque markers in a coordinate system of the fluoroscope.
 12. Themethod according to claim 9, wherein creating a Gaussian mask includescreating a Gaussian mask by defining an acceptable range of movementaround the determined locations of the radio opaque markers.
 13. Themethod according to claim 12, wherein the acceptable range of movementis an acceptable range of movement in both an X and a Y direction in thereference frame.
 14. The method according to claim 9, whereindetermining whether the fluoroscope has moved includes determiningwhether the position of the radio opaque markers is outside of theGaussian mask. 15-20. (canceled)
 21. The method according to claim 3,wherein calculating the expected interval between radio opaque markersincludes calculating the expected interval between the radio opaquemarkers at the stored height and angle of the fluoroscope at which thereference frame was captured.
 22. A system for detecting movement of afluoroscope, comprising: a fluoroscopic imaging device; a plurality ofradio opaque markers disposed in a reference frame of the fluoroscope; aworkstation in operative communication with the fluoroscope, theworkstation including a memory and at least one processor, the memorystoring instructions, which when executed by the processor, isconfigured to: capture a fluoroscopic reference frame; calculate aninterval between the radio opaque markers in the reference frame;calculate an expected interval between radio opaque markers for aplurality of levels of zoom; and identify the expected interval thatmost closely matches the calculated interval in the reference frame. 23.The system according to claim 22, wherein the workstation is furtherconfigured to store a height of the fluoroscope from which thefluoroscopic reference frame is captured compared to a location of theradio opaque markers.
 24. The system according to claim 23, wherein theworkstation is configured calculate the expected interval between theradio opaque markers at the stored height of the fluoroscope at whichthe reference frame is captured.
 25. The system according to claim 24,wherein the workstation is further configured to apply a transform suchthat a location of a target can be accurately overlaid on a livefluoroscopic image.
 26. The system according to claim 22, wherein theworkstation is configured to capture the fluoroscopic reference framewhen the fluoroscope is in an anteroposterior position.
 27. The systemaccording to claim 22, wherein the workstation is configured to capturethe fluoroscopic reference frame through a sweep through about 30-60degrees relative to an anteroposterior position.